Conversation
Add completable() wrappers to workflow prompt parameters so VS Code Copilot Chat shows auto-complete dropdowns for language, query paths, SARIF files, database paths, and pack roots. - Create prompt-completions.ts with completion providers for each parameter type (language enum, .ql/.qlref files, .sarif files, CodeQL databases, codeql-pack.yml directories) - Update all 14 workflow prompt registrations to use addCompletions() - Add comprehensive unit tests (35 tests covering all completers and the addCompletions utility) Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/e316ece4-06c5-45d7-8020-062e6ec67e39 Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
- cloneStringType now accepts both ZodString and ZodEnum (for prompts that use raw schema.shape without toPermissiveShape) - Add runtime type checks that throw clear errors for unexpected types - Add 2 unit tests for ZodEnum handling in addCompletions Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/e316ece4-06c5-45d7-8020-062e6ec67e39 Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/e316ece4-06c5-45d7-8020-062e6ec67e39 Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
Dependency Review✅ No vulnerabilities or license issues or OpenSSF Scorecard issues found.Snapshot WarningsEnsure that dependencies are being submitted on PR branches and consider enabling retry-on-snapshot-warnings. See the documentation for more information and troubleshooting advice. Scanned FilesNone |
Addresses three UX issues with MCP prompt argument completions: 1. Database discovery: completeDatabasePath now scans $HOME/codeql/databases/ as a default search location, and recursively discovers directories containing codeql-database.yml (including .testproj dirs) in the workspace. 2. Language auto-derivation: For prompts where queryPath is provided, language is now optional and auto-derived from the nearest codeql-pack.yml's codeql/<lang>-all or codeql/<lang>-queries dependency. Schema field ordering now puts queryPath before language so VS Code prompts for the query first. Affected schemas: explainCodeqlQuerySchema, documentCodeqlQuerySchema, workshopCreationWorkflowSchema, qlLspIterativeDevelopmentSchema. 3. Query path completion filtering: completeQueryPath now skips .github, dist, coverage, and build directories in addition to node_modules, .git, and .tmp — preventing workshop examples and build artifacts from cluttering the VS Code dropdown. New tests: 19 unit tests covering all three fixes. Updated tests: 9 existing tests updated to reflect language being optional.
There was a problem hiding this comment.
Pull request overview
Adds MCP prompt-argument completions so VS Code Copilot Chat slash commands can provide dropdown suggestions (languages, query paths, SARIF paths, databases, pack roots), and updates several prompt schemas/handlers to allow omitting language with best-effort auto-derivation from codeql-pack.yml. The PR also includes a set of VS Code extension-side changes around CodeQL CLI discovery and pack installation logging, plus new extension E2E coverage for completion/complete.
Changes:
- Introduce
server/src/prompts/prompt-completions.tswith completion providers andaddCompletions()to wrap prompt shapes with MCP SDKcompletable(). - Update workflow prompt registrations to apply
addCompletions()and (for some prompts) makelanguageoptional + attempt auto-derivation from pack metadata. - Add unit tests (server) and E2E integration tests (VS Code extension) for completion behavior, alongside additional extension improvements/tests for CLI resolution and pack installer logging.
Show a summary per file
| File | Description |
|---|---|
| server/src/prompts/prompt-completions.ts | New completion providers + addCompletions() wrapper; also adds language auto-derivation helper. |
| server/src/prompts/workflow-prompts.ts | Wrap prompt registration shapes with addCompletions(); make language optional for several schemas; add handler-side language auto-derivation + warnings. |
| server/test/src/prompts/prompt-completions.test.ts | New unit tests for completion providers, schema wrapping behavior, and language auto-derivation from pack metadata. |
| server/test/src/prompts/workflow-prompts.test.ts | Update schema expectation tests to reflect language becoming optional in select prompts and updated handler messaging. |
| server/dist/codeql-development-mcp-server.js | Updated bundled build output reflecting server-side changes. |
| extensions/vscode/test/suite/mcp-completion-e2e.integration.test.ts | New Extension Development Host E2E tests calling completion/complete against the running MCP server. |
| extensions/vscode/test/suite/mcp-server.integration.test.ts | Adds an environment validation test for CODEQL_PATH (currently substring-based). |
| extensions/vscode/esbuild.config.js | Registers the new extension integration test entry point for compilation. |
| extensions/vscode/src/codeql/cli-resolver.ts | Adds in-flight resolve de-duplication and validates PATH-resolved binaries before accepting them. |
| extensions/vscode/test/codeql/cli-resolver.test.ts | Adds tests for PATH-based version detection and concurrent resolution behavior. |
| extensions/vscode/src/server/pack-installer.ts | Enhances logging (CLI vs target version), adds success counts, and changes terminology to “install” for pack download. |
| extensions/vscode/test/server/pack-installer.test.ts | Adds tests asserting updated pack installer logging and summary behavior. |
| extensions/vscode/src/server/server-manager.ts | Tweaks “bundled server” log message text. |
| extensions/vscode/test/server/server-manager.test.ts | Updates expectations to match new server-manager logging and asserts no “npm install” mention. |
Copilot's findings
Comments suppressed due to low confidence (3)
server/src/prompts/workflow-prompts.ts:348
- The
explainCodeqlQuerySchemaparameter comment still saysqueryPathandlanguageare required, butlanguageis now.optional(). Please update the comment so it matches the schema/handler behavior (language may be omitted and auto-derived).
* - `databasePath` is optional – a database may also be derived from tests.
*/
export const explainCodeqlQuerySchema = z.object({
queryPath: z
.string()
.describe('Path to the CodeQL query file (.ql or .qlref)'),
language: z
.enum(SUPPORTED_LANGUAGES)
.optional()
.describe('Programming language of the query (auto-derived from pack metadata when omitted)'),
server/src/prompts/workflow-prompts.ts:367
- The
documentCodeqlQuerySchemacomment saysqueryPathandlanguageare required, butlanguageis now optional. Please update the comment to reflect the new validation contract (and the handler’s auto-derivation/warning behavior).
/**
* Schema for document_codeql_query prompt parameters.
*
* - `queryPath` and `language` are **required**.
*/
export const documentCodeqlQuerySchema = z.object({
queryPath: z
.string()
.describe('Path to the CodeQL query file (.ql or .qlref)'),
language: z
.enum(SUPPORTED_LANGUAGES)
.optional()
.describe('Programming language of the query (auto-derived from pack metadata when omitted)'),
server/src/prompts/workflow-prompts.ts:454
qlLspIterativeDevelopmentSchemanow haslanguageoptional, but the schema comment still stateslanguageandqueryPathare required. Please update the comment to match the schema (language omitted -> auto-derive + warning).
/**
* Schema for ql_lsp_iterative_development prompt parameters.
*
* - `language` and `queryPath` are **required** – LSP tools need both.
* - `workspaceUri` is optional – defaults to the pack root.
*/
export const qlLspIterativeDevelopmentSchema = z.object({
queryPath: z
.string()
.describe('Path to the query file being developed'),
language: z
.enum(SUPPORTED_LANGUAGES)
.optional()
.describe('Programming language for the query (auto-derived from pack metadata when omitted)'),
- Files reviewed: 13/15 changed files
- Comments generated: 9
File watcher events (database discovery, query result creation) were triggering envBuilder.invalidate() + mcpProvider.fireDidChange(), causing VS Code to re-query provideMcpServerDefinitions() on every workspace file change. The debounce only coalesced rapid events but did not prevent the unnecessary re-provision cycle. The MCP server definition only needs to change when: - The extension itself changes (update/reinstall) - Workspace folder registration changes (folders added/removed) - Configuration changes that affect the server The running server discovers databases and query results on its own through filesystem scanning at tool invocation time, so notifying VS Code of file content changes is unnecessary. Also includes: - Prompt completion caching with 5s TTL to avoid repeated filesystem scans during rapid completion requests - Debounce timer cancellation when McpProvider is disposed - Unit tests verifying watchers do not trigger fireDidChange - Integration tests verifying file creation does not fire onDidChangeMcpServerDefinitions
ql-mcp VS Code extension UX
There was a problem hiding this comment.
Pull request overview
This PR improves the ql-mcp VS Code extension and server prompt UX by adding MCP prompt argument completions, refining watcher/restart behavior, strengthening CLI resolution robustness, and expanding automated test coverage.
Changes:
- Add prompt argument completion providers (language + file/path discovery) and wire them into workflow prompt registrations.
- Make several workflow prompt
languageparameters optional with auto-derivation from CodeQL pack metadata and updated handler messaging/tests. - Improve extension-side robustness/UX via CLI resolution concurrency control, debounced MCP definition change notifications, clearer logging, and new unit/integration tests.
Show a summary per file
| File | Description |
|---|---|
server/src/prompts/prompt-completions.ts |
Adds completion callbacks + schema enhancement via completable() and language auto-derivation from pack metadata. |
server/src/prompts/workflow-prompts.ts |
Makes language optional for several prompts, derives language from pack metadata, and applies addCompletions() to prompt shapes. |
server/test/src/prompts/prompt-completions.test.ts |
Adds unit tests covering completion behavior, caching, schema enhancement, and pack-language derivation. |
server/test/src/prompts/workflow-prompts.test.ts |
Updates schema/handler tests for optional language and updated warning behavior. |
server/dist/codeql-development-mcp-server.js |
Updates bundled build artifact to include the new prompt completion functionality and schema changes. |
extensions/vscode/src/codeql/cli-resolver.ts |
Adds in-flight resolution deduplication and validation for PATH-discovered CLI binaries. |
extensions/vscode/test/codeql/cli-resolver.test.ts |
Adds tests for PATH version detection, validation failure fallback, and concurrent resolution behavior. |
extensions/vscode/src/server/mcp-provider.ts |
Debounces fireDidChange() notifications and cancels pending debounce on restart. |
extensions/vscode/test/server/mcp-provider.test.ts |
Adds unit tests validating debounce/coalescing behavior and restart cancellation semantics. |
extensions/vscode/src/extension.ts |
Stops invalidating/re-providing MCP definitions on file-content watcher events; keeps watchers for logging/state only. |
extensions/vscode/test/extension.test.ts |
Adds tests ensuring watcher events do not trigger MCP definition updates or env invalidation. |
extensions/vscode/src/server/server-manager.ts |
Clarifies bundled server log message (“Bundled server ready”). |
extensions/vscode/test/server/server-manager.test.ts |
Updates assertions to match the new bundled server log wording and ensures no “npm install” mention. |
extensions/vscode/src/server/pack-installer.ts |
Improves logging around CLI/target versions, download/install flow, and per-language success summaries. |
extensions/vscode/test/server/pack-installer.test.ts |
Adds tests for new logging behaviors and summary counts. |
extensions/vscode/src/bridge/database-watcher.ts |
Logs discovered/removed databases using relative paths for readability. |
extensions/vscode/src/bridge/query-results-watcher.ts |
Logs created BQRS/SARIF results using relative paths for readability. |
extensions/vscode/__mocks__/vscode.ts |
Extends workspace mock with asRelativePath and updateWorkspaceFolders for tests. |
extensions/vscode/test/suite/mcp-server.integration.test.ts |
Adds an integration test asserting CODEQL_PATH shape when available. |
extensions/vscode/test/suite/mcp-completion-e2e.integration.test.ts |
Adds end-to-end tests that exercise MCP completion/complete against the real server in the extension host. |
extensions/vscode/test/suite/file-watcher-stability.integration.test.ts |
Adds integration tests ensuring file changes do not trigger redundant MCP definition change events. |
extensions/vscode/esbuild.config.js |
Registers the new VS Code integration test entry points for esbuild compilation. |
Copilot's findings
Comments suppressed due to low confidence (3)
server/src/prompts/workflow-prompts.ts:1034
- The warning advises ensuring a
codeql/<lang>-alldependency, butresolveLanguageFromPack()also resolves languages fromcodeql/<lang>-queries. To avoid misleading guidance, update the message to mention both-alland-queries(or align the resolver + message to the same rule).
if (!effectiveLanguage) {
warnings.push('⚠ **Language could not be auto-derived.** Please provide the `language` parameter or ensure the query is inside a CodeQL pack with a `codeql/<lang>-all` dependency.');
}
server/src/prompts/workflow-prompts.ts:1098
- The warning advises ensuring a
codeql/<lang>-alldependency, butresolveLanguageFromPack()also resolves languages fromcodeql/<lang>-queries. To avoid misleading guidance, update the message to mention both-alland-queries(or align the resolver + message to the same rule).
if (!effectiveLanguage) {
warnings.push('⚠ **Language could not be auto-derived.** Please provide the `language` parameter or ensure the query is inside a CodeQL pack with a `codeql/<lang>-all` dependency.');
}
server/src/prompts/workflow-prompts.ts:1215
- The warning advises ensuring a
codeql/<lang>-alldependency, butresolveLanguageFromPack()also resolves languages fromcodeql/<lang>-queries. To avoid misleading guidance, update the message to mention both-alland-queries(or align the resolver + message to the same rule).
if (!effectiveLanguage) {
warnings.push('⚠ **Language could not be auto-derived.** Please provide the `language` parameter or ensure the query is inside a CodeQL pack with a `codeql/<lang>-all` dependency.');
}
- Files reviewed: 21/23 changed files
- Comments generated: 5
There was a problem hiding this comment.
Pull request overview
This PR improves the ql-mcp VS Code extension and MCP server prompt UX by adding prompt-argument completions (for better slash-command input), making several workflow prompt parameters more robust (e.g., optional language with auto-derivation), and reducing noisy/redundant extension definition-change events via watcher behavior changes and debouncing.
Changes:
- Add MCP prompt argument completion providers (language + workspace file/path discovery) and wire them into workflow prompt registration.
- Make
languageoptional for several workflow prompts and auto-derive it fromcodeql-pack.ymlwhen possible, emitting warnings when it can’t be derived. - Improve VS Code extension robustness: CLI resolution concurrency/validation, pack install/download logging, watcher behavior, and debounced MCP definition-change notifications; expand unit/integration test coverage accordingly.
Show a summary per file
| File | Description |
|---|---|
| server/test/src/prompts/workflow-prompts.test.ts | Updates prompt schema/handler tests to reflect optional language and new warning behavior. |
| server/test/src/prompts/prompt-completions.test.ts | Adds unit tests for completion providers, caching, directory filtering, and pack-language derivation. |
| server/test/src/prompts/constants.test.ts | Adds tests ensuring shared SUPPORTED_LANGUAGES constant is exported and sorted. |
| server/src/prompts/workflow-prompts.ts | Makes language optional for select prompts, auto-derives language from pack metadata, and attaches completion metadata to prompt schemas. |
| server/src/prompts/prompt-completions.ts | Introduces completion providers (language/query/db/sarif/packRoot) plus schema enhancement via completable(). |
| server/src/prompts/constants.ts | Moves shared prompt constants (supported languages) to a dedicated module to avoid circular deps. |
| server/dist/codeql-development-mcp-server.js | Updates bundled server output to include new prompt completion and prompt behavior changes. |
| extensions/vscode/test/suite/mcp-server.integration.test.ts | Adds an integration assertion around CODEQL_PATH shape when present. |
| extensions/vscode/test/suite/mcp-completion-e2e.integration.test.ts | Adds end-to-end tests that invoke MCP completion/complete for prompt arguments. |
| extensions/vscode/test/suite/file-watcher-stability.integration.test.ts | Adds integration coverage ensuring file changes don’t spuriously trigger definition updates. |
| extensions/vscode/test/server/server-manager.test.ts | Updates expectations for revised “bundled server ready” logging. |
| extensions/vscode/test/server/pack-installer.test.ts | Adds tests for new pack install/download logging and success-count summaries. |
| extensions/vscode/test/server/mcp-provider.test.ts | Adds unit tests for debounced fireDidChange, restart behavior, and disposal. |
| extensions/vscode/test/extension.test.ts | Adds tests ensuring watcher events don’t trigger environment invalidation or definition change events. |
| extensions/vscode/test/codeql/cli-resolver.test.ts | Adds tests for PATH validation/version detection, concurrent resolution, and invalidation of in-flight resolve. |
| extensions/vscode/src/server/server-manager.ts | Refines bundled-server readiness log message for clarity. |
| extensions/vscode/src/server/pack-installer.ts | Enhances logging (CLI/target versions, download vs bundled install, per-language success counts). |
| extensions/vscode/src/server/mcp-provider.ts | Debounces fireDidChange, cancels pending debounce on restart/dispose. |
| extensions/vscode/src/extension.ts | Stops treating watcher file events as a trigger for MCP server definition changes (logging/state only). |
| extensions/vscode/src/codeql/cli-resolver.ts | Prevents duplicate concurrent CLI resolutions, validates PATH results, and discards stale in-flight results after invalidation. |
| extensions/vscode/src/bridge/query-results-watcher.ts | Logs watcher events using relative paths for readability. |
| extensions/vscode/src/bridge/database-watcher.ts | Logs discovered/removed databases using relative paths for readability. |
| extensions/vscode/esbuild.config.js | Adds new integration test entry points to the extension test build config. |
| extensions/vscode/mocks/vscode.ts | Extends VS Code workspace mock with asRelativePath and updateWorkspaceFolders. |
Copilot's findings
- Files reviewed: 23/25 changed files
- Comments generated: 3
|
@copilot apply changes based on the comments in this thread |
…EffectiveLanguage helper, add Windows env stubs - Centralize directory skip list into SKIP_DIRS set used by findFilesByExtension, findDatabaseDirs, and completePackRoot (adds dist/, coverage/, build/, .github/ to completePackRoot) - Extract getEffectiveLanguage() helper to eliminate duplicated language auto-derivation logic across 4 prompt handlers - Add Windows env var stubs (HOMEDRIVE, HOMEPATH, USERPROFILE) to database completion tests for cross-platform compatibility - Add 5 unit tests for getEffectiveLanguage() Agent-Logs-Url: https://github.com/advanced-security/codeql-development-mcp-server/sessions/e315dcb4-ed12-428b-8294-14b398746c4c Co-authored-by: data-douser <70299490+data-douser@users.noreply.github.com>
All three review comments addressed in
|
Closes #229.
Summary of Changes
This PR introduces several improvements to the VS Code extension for the
ql-mcpserver -- focusing on robustness, logging clarity, and test coverage in order to improve the overall user experience (UX) with the CodeQL MCP VS Code extension. The most significant changes include enhanced concurrency handling and validation in CLI resolution, improved logging and debouncing for server and pack operations, and expanded test cases. Additionally, file path logging is now more user-friendly, and the test infrastructure is strengthened for watcher events.Outline of Changes
CLI Resolution and Robustness Improvements:
CliResolverto ensure only one resolution is in progress at a time, preventing duplicate work and race conditions. Added aresolvePromisefield and split logic into a newdoResolvemethod. [1] [2]Logging and Diagnostics Enhancements:
PackInstallerto provide detailed status messages about detected CLI versions, pack downloads, and bundled pack installation results, including per-language success counts. [1] [2] [3] [4]DatabaseWatcherandQueryResultsWatcherto use relative paths for improved readability. [1] [2] [3]Server Notification and Watcher Behavior:
McpProvider.fireDidChangeto coalesce rapid-fire notifications into a single event, and ensured restarts cancel pending notifications. [1] [2] [3]Testing and Mocks:
DatabaseWatcherandQueryResultsWatcherto support manual firing of change events. [1] [2]API and Utility Updates:
asRelativePathandupdateWorkspaceFoldersmocks to the VS Code mock workspace API.These changes collectively improve the extension’s reliability, observability, and testability.